This invention relates to retinal display devices, and more particularly to a method and apparatus for mapping and tracking a viewer""s eye.
A retinal display device is an optical device for generating an image upon the retina of an eye. Light is emitted from a light source, collimated through a lens, then passed through a scanning device. The scanning device defines a scanning pattern for the light. The scanned light converges to focus points on an intermediate image plane. As the scanning occurs the focus point moves along the image plane (e.g., in a raster scanning pattern). The light then diverges beyond the plane. An eyepiece is positioned along the light path beyond the intermediate image plane at some desired focal length. An xe2x80x9cexit pupilxe2x80x9d occurs shortly beyond the eyepiece in an area where a viewer""s eye pupil is to be positioned.
A viewer looks into the eyepiece to view an image. The eyepiece receives light that is being deflected along a raster pattern. Modulation of the light during the scanning cycle determines the content of the image. For a see-through virtual retinal display a user sees the real world environment around the user, plus the added image of the display projected onto the retina.
A viewer wearing a head-mounted virtual retinal display typically moves their eye as they look at images being displayed. According to the invention, the direction the viewer looks is tracked with the display. Prior to tracking, a map of the viewer""s eye is generated by the display. The map includes xe2x80x98landmarksxe2x80x99 such as the viewer""s optic nerve, fovea, and blood vessels. Thereafter, the relative position of one or more landmarks is used to track the viewing direction. The head-mounted display includes a light source and a scanner. The scanner deflects light received from the light source to scan a virtual image onto a viewer""s retina in a periodic manner. During each scanning period, light is deflected along a prescribed pattern. To generate a map, and thereafter to monitor viewing direction, light reflected off the viewer""s retina is monitored. Some of the reflected light travels back into the display device. The content of the reflected light will vary depending upon the image light projected and the features of the viewer""s retina. During the initial mapping stage, the content of the image light can be fixed at a constant intensity, so that the content of the reflected light is related only to the feature""s (i.e., landmarks) of the retina. The changing content of the reflected light is sampled at a sampling rate and stored. The scanner position at the time of each sample is used to correlate a position of the sample. The relative position and the content represent a map of the viewer""s retina.
According to one aspect of the invention, the light reflected from the viewer""s eye travels back into an eyepiece and along a light path within the retinal display device. In a specific embodiment the reflected light is deflected by the scanner toward a beamsplitter. The beamsplitter deflects the reflected light toward a photodetector which samples the reflected light content. The beamsplitter is positioned between the light source and the scanner of the retinal display device.
For generating a virtual image, light emitted from the light source passes through the beamsplitter to the scanning subsystem and onward to the eyepiece and the viewer""s eye. Light reflected from the viewer""s eye passes back along the same path but is deflected so as not to return to the light source. Instead the light is deflected toward the photodetector. Thus, the beamsplitter passes light which is incident in one direction (e.g., light from the light source) and deflects light which is incident in the opposite direction (e.g., reflected light from the viewer""s eye).
According to another aspect of the invention, a specific feature of the retina (e.g., fovea position) is monitored over time to track where the viewer is looking (i.e., the viewer""s center of vision). The landmarks in the retina which correspond to such feature will cause the reflected light to exhibit an expected pattern. The relative position of such pattern in the reflected light will vary according to the viewing direction. By identifying the pattern and correlating the relative orientation of the pattern to the orientation of the corresponding feature in the map, the change in viewing direction is determined. In various applications, such position indication is used as a pointing device or is used to determine image content. For example, as a pointing device the fovea position indicates pointer position. A blink of the eye for example, corresponds to actuating a pointing device (e.g., xe2x80x9cclickingxe2x80x9d a computer mouse.)
According to another aspect of the invention, the map of the viewer""s retina is stored and used for purposes of viewer identification. In a security application for example, a viewer is denied access to information or denied operation of a computer or display when the viewer""s retina does not correlate to a previously stored map of an authorized user.